Overview of the ImageCLEFmed 2008 Medical Image Retrieval Task

2008 was the fifth year for the medical image retrieval task of ImageCLEF, one of the most popular tracks within CLEF. Participation continued to increase in 2008. A total of 15 groups submitted 111 valid runs. Several requests for data access were also received after the registration deadline. The most significant change in 2008 was the use of a new database containing images from the medical literature. These images, part of the Goldminer collection, were from the RSNA journals Radiology and Radiographics. Besides the images, the figure captions and the part of the caption referring to a particular sub figure were supplied to the participants. Access to the full text articles in HTML was also provided, as was each article’s Medline PMID (PubMed Identifier). An article’s PMID could be used to obtain the officially assigned MeSH (Medical Subject Headings) terms. Unlike previous years, this year’s collection was entirely in English, as it was obtained from English-language medical literature. However, the topics were, as in previous years, supplied in German, French, and English. The topics used in 2008 were a subset of the 85 topics used in 2005-2007. Thirty topics were made available, ten in each of three categories: visual, mixed, and semantic. As in previous years, most groups concentrated on fully automatic retrieval. However, three groups submitted a total of seven manual or interactive runs; these runs did not show a substantial increase in performance over the automatic approaches. In previous years, multi–modal combinations were the most frequent submissions. However, in 2008 only half as many mixed runs as purely textual runs were submitted. Very few fully visual runs were submitted, and the ones submitted performed poorly. This may be explained in part by the heavily semantic nature of the 2008 topics. The best MAP scores were very similar for textual and multi–modal approaches, whereas early precision performance was clearly better for the multi-modal approaches.